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Abstract 

The purpose of this study was to examine the comparative influence of college administrator credential 
programs, on-the-job experiences, and the ISLLC Standards in the development of leadership expertise 
among urban public school principals. An exploratory, ex-post-facto research design used both quanti¬ 
tative and qualitative approaches. A survey of 101 randomly selected urban school principals from 25 of 
America’s largest metropolitan school districts was given, followed by telephone interviews with a subset 
of 20 randomly selected survey participants. Data were analyzed through the use of inferential and de¬ 
scriptive statistics and descriptive narratives. On-the-job experiences were significantly more important 
in developing leadership expertise than college credential programs on each of 41 ISLLC-based learn¬ 
ing tasks. However, college credential programs are also important sources of leadership development. 
Several significant differences were found between comparative ratings of learning tasks by subgroups 
(experience, gender, school type). Respondents ranked 78% of the ISLLC Standards as very impor¬ 
tant to the field of school leadership. Interview subjects expressed concern about lack of preparation in 
budgeting, data analysis, teacher evaluation, and change management. 
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2INTRODUCTON 

What school leaders do and the impact of their behaviors on teaching and learning has been the subject of 
scholarly inquiry for decades (Murphy, 1990; Tyack & Hansot, 1982), while the magnitude of the principal’s 
role in contemporary school reform efforts has become a topic of great interest by policy makers, scholars, 
and practitioners (Hess & Kelly, 2005). New policy initiatives such as Race To the Top underscore on 
the importance of the principal’s role in promoting school reforms and, subsequently, increased student 
achievement. According to United States Department of Education Secretary, Arne Duncan (2010, p.l), 
Race to the Top “is driving a deep rethinking of education” and is pushing for “all elements of the education 
system to get better.. .simultaneously.” In his remarks to the National Education Association Duncan 
(2009, p. 4) further stated that, “Great principals lead talented instructional teams that drive student 
performance and close achievement gaps.. .But if they’re not up to the job, they need to go.” In addition 
to holding principals accountable for student achievement, Race to the Top also underscores a significant 
commitment by the Department of Education to provide funding support for effective principal preparation 
and professional development programs. 

Recent analyses regarding programs and practices that prepare school leaders have illuminated both 
promising developments and concerns regarding the efficacy, quality, and relevance of principal preparation 
programs as well as the principals produced by them. To date, the “scorecard” is mixed. Critics point to the 
dismal state of principal preparation programs in America, while proponents point to important advances 
in the professional knowledge base, policy development, and local practices in school leadership (Davis, 
Darling-Hammond, LaPointe, & Meyerson, 2006; Murphy & Vriesenga, 2006; Williams, Kirst, & Haertel, 
2005). 

1 http://www.ncpeapublications.org/latest-issue-ijelp.html 
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Some critics have concluded that most principal preparation programs are simply not very good at 
producing leaders who can address the increasingly complex challenges and problems faced by America’s 
public schools today, and especially those faced by schools in low income, urban communities (Elmore, 2000; 
Hess & Kelly, 2005; Levine, 2005). According to Fry, O’Neill, and Bottoms (2006, p. 7), “It is difficult to 
distinguish most of the programs of today from those of the preceding generation.” 

School district superintendents have expressed similar concerns. In a survey of 853 superintendents, 
Farkas, Johnson, Duffett, Foleno, and Foley (2001) found that ninety-two percent believed that administrator 
credential programs are out of touch and ineffective. Even school principals generally agree that the “factors 
that add the greatest value to their success [as leaders] are on-the-job experiences” (Usdan, McCloud, & 
Podmostko, 2000, p. 1). 

In contrast to such negative accounts is growing evidence that educational leadership is a profession in 
transition, moving toward more robust approaches to leadership preparation (e.g., standards based princi¬ 
pal preparation programs, programs designed to reform underperforming urban schools, and accountability 
measures to foster leadership effectiveness). However, while the need for highly effective school principals is 
indisputable, understanding how such principals are developed is imperative if the profession is to effectively 
meet the complex needs of 21 st Century schools and students. 

In this study, we draw upon the standards of professional practice established by the Interstate School 
Leadership Licensure Consortium to examine, a) how urban school principals develop leadership expertise, 
b) the comparative importance of administrator credential programs and on-the-job experiences as sources 
of leadership knowledge and skill development and, c) the relative importance of the ISLLC standards to 
urban principals and their impact on leadership development. 

3 CONCEPTUAL FRAMEWORK 

3.1 School Leadership Development Under Fire 

The increased attention on school leadership and its potential to facilitate powerful teaching and learning 
(Leithwood, Seashore-Louis, Anderson, & Wahlstrom, 2004; Marzano, Waters, & McNaulty, 2005); Seashore 
Louis, Leithwood, Wahlstrom, & Anderson, 2010) has stimulated policy makers, researchers, credential 
program faculty, practitioners, non-profit agencies, and foundations to establish more relevant, rigorous, and 
innovative professional training programs and performance standards (e.g., the Wallace Foundation, USDE 
Leadership Development Grants, ISLLC 2008, etc.). In some states, like California, legislative efforts to 
tap into the pool of administrative talent from non-public school, private, and public employment sectors 
have resulted in alternative pathways for administrative licensure- such as a testing only option, relaxed 
credential requirements for charter school principals, credentialing offered through local education agencies 
and professional associations, and the inclusion of non-profit credential providers such as New Leaders 
for New Schools (Darling-Hammond, LaPointe, Meyerson, Orr, & Cohen, 2007; Hale & Moorman, 2003; 
Hamilton, Stecher, & Yuan, 2009; Levin, 2005; Murphy, 2003). To date, little empirical evidence points 
to the comparative advantage of either alternative or traditional credential programs in preparing effective 
principals. According to Hess and Kelly (2005, p.175), the “world of principal preparation shows little 
evidence that the whirlwind of initiatives and new programs has yielded much in the way of substantive 
change.” The authors maintain that with the exception of New Leaders for New Schools (NLNS) and the 
Knowledge is Power Program (KIPP), most programs depicted as innovative or alternative have simply 
“continued conventional practices repackaged under the mantle of reform” (p. 175). 

Nevertheless, at the heart of the reformists’ agenda are several core competencies deemed essential to 
effective instructional leadership (Darling-Hammond, et ah, 2007; Hallinger, 1992; Hallinger & Heck, 1998; 
ISLLC, 1996; Leithwood, Seashore-Louis, Anderson, & Wahlstrom, 2004; Marks & Printy, 2003; Murphy, 
2003; Silins, Mulford, & Zarins, 2002). Stanford professor, Larry Cuban (2004), maintains that urban 
principals and the students they serve often face stiffer challenges politically, socially, and economically than 
their more affluent suburban counterparts. According to Cuban, the most successful urban principals share 
four common attributes, a) a refusal to accept low expectations, b) a focus on instructional excellence, c) an 
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insistence on providing a challenging curriculum, and d) the ability to rally broad support politically. 

Although much has been written about the characteristics of effective urban principals the activities and 
settings through which they acquire leadership skills are not well understood. According to Murphy (2003, 
p. 30), “Leadership is a complex and context-dependent activity. To attempt to envelop the concept with a 
definitive list of indicators is a fool’s errand.” Whereas the traditional research orientation has emphasized 
leadership program inputs (e.g., resources, program structure, curriculum design), many scholars maintain 
that a more comprehensive orientation would place a stronger emphasis on the examination of program 
throughputs (e.g., learning processes and activities) and their outcomes (Fry, O’Neill, & Bottoms, 2006; 
Hale & Moorman, 2003; McGough, 2002; Murphy & Vriesenga, 2006). 

Often, assessments of principal preparation programs have less to do with what principals do (and how 
well) once employed in their new leadership assignments than the curricular and structural characteristics 
of the credential programs themselves (Elmore, 2000; Hess & Kelly, 2005; Orr, 2003;). Some critics assert 
that too many preparation programs fail to teach principals how to, a) be instructional, community, and 
visionary leaders, b), effectively meet the needs of diverse learners, or c) use multiple sources of data to 
diagnose and repair ineffective educational systems and practices, or d) provide comprehensive and hands-on 
field experiences (Hess & Kelly, 2005, Usdan, McCloud, & Podmostko, 2000). Arthur Levine (2005) argued 
that if indeed “the proof was in the pudding,” most principal preparation programs in American would fail 
(and have failed) miserably. Usdan, et ah, (2000, p. 2) states that “.. .current principals find very little in 
their professional preparation or ongoing professional development to equip them for new roles.” 

Concerns like these underscore an important question that continues to challenge the field: what do 
urban principals perceive to be the most influential venue for developing leadership expertise, the job or 
administrative credential programs? In response to this, we focused our investigation on the perceptions of 
urban principals to three sub-questions: 

1. How do urban principals develop leadership expertise? 

2. How important (comparatively) are on-the-job experiences and administrative credential programs in 
the development of leadership expertise among urban principals (and with what skills)? 

3. How important are the 1996 ISLLC Standards for School Leaders in the development of leadership 
expertise among urban principals? 

We believe that the answers to these questions must be framed within the larger policy context of principal 
preparation and professional development in America a context that has been profoundly shaped by the 
ISLLC Standards for School Leaders. 

3.2 The National Standards for Administrative Practice 

In response to growing pressures to reform the quality of America’s schools and their leaders, in 1994 the 
Council of Chief State School Officers (CCSSO) commissioned the Interstate School Leadership Licensure 
Consortium (ISLLC) to shape and focus the development of school leaders around a set of empirically 
grounded principles and best practices. The resulting document (i.e., the ISLLC Standards for School 
Leaders) subsequently became the “gold standard” which most states now use to shape K-12 administrator 
credentialing requirements and performance criteria (Murphy, 2003, Usdan, et al., 2000). For over 12 years 
the standards remained intact. Then, in 2008 the standards were revised to strengthen their alignment 
with the evolving educational and leadership needs of 21 st Century schools. Nevertheless, the original six 
standards have had a growing influence on programs that prepare school leaders. It is our position that the 
analysis of how school leaders develop leadership expertise must recognize the broad and enduring influence 
of the ISLLC Standards, and particularly those developed in 1996 (e.g., the revised version will require more 
time to penetrate the field). 

The 1996 ISLLC Standards are organized into three domains, six conceptual categories, and 184 sup¬ 
porting concepts-43 (23%) that fall within the domain of knowledge , 44 (24%) that fall within the domain 
of dispositions, and 97 (53%) that fall within the domain of performances (ISLLC, 1996). Importantly, the 
ISLLC Standards are widely regarded as empirically sound and have garnered broad support from policy 
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makers, scholars, and practitioners. As such, they provide a strong framework for examining the mechanisms 
and contexts that influence the acquisition of leadership expertise (Murphy, 2003; Usdan, et ah, 2000). 

3.3 Experiential Learning and Principal Development 

“Most of what I learned about leadership, I learned on the job.” This is a statement often heard from vet¬ 
eran school administrators, but how accurate is it? According to ninety-eight percent of the principals who 
responded to a nationally distributed survey by Hess & Kelly (2005), learning on the job was far more impor¬ 
tant in the development of leadership skills than graduate school coursework. That people learn about (and 
from) their jobs while performing them is not a new idea. It is common knowledge among those who work 
with adult learners that formal classroom instruction is no substitute for the real world of work in the devel¬ 
opment of professional expertise (Bottoms & O’Neill, 2001; Young, 2002). Scholars have studied experiential 
learning, situated cognition, job-embedded learning, and transformative learning in the workplace for years 
(Fenwick, 2003; Kolb, Boyatszis, & Mainemelis, 2000; Lave & Wenger, 1991; Mezirow, 1991). In the business 
sector, leadership development that occurs in the context of ongoing work activities and tied to strategic 
organizational initiatives is now considered state of the art (Felstead, Fuller, Jewson, & Unwin, 2009). How¬ 
ever, this is not commonly the case in education (Hess & Kelly, 2005). In fact, there is little research on how 
on-the-job experiences can be used as a purposeful and structured professional development tool for school 
leaders, or how they can stimulate transformative learning (i.e., experiences that when combined with deep 
self reflection and social discourse challenge deeply held assumptions, values, and beliefs—thus facilitating 
learning and new knowledge structures) (Fenwick, 2003; Mezirow, 1991). These are important issues since 
educators rely on job-embedded learning to improve instructional practices and leadership skills, but often 
without full recognition of what they are doing (Wood & Killian, 1998). As a result, learning on the job 
is often a tacit rather than explicit process and may rely more on serendipitous events than deliberately 
structured activities and carefully designed evaluative processes (Wood & McQuarrie, 1999). 

4 A Rationale for Comparing the Influence of On-the-Job Experiences and Grad¬ 
uate Credential 

4.1 Programs in the Development of Leadership Expertise 

A deeper understanding of the comparative influence of on-the-job experiences and graduate credential 
programs in the development of leadership expertise is important for several reasons: 

1. Since approximately 90% of all administrators in the United States are certified through graduate school 
coursework, the efficacy of such programs is clearly an important issue. If it is true that for most school 
leaders on-the-job experiences are more important than graduate coursework in the development of 
leadership skills, then the relevance, design, and conduct of graduate programs (as well as the costs) 
are indeed cause for deep concern (Hess & Kelly, 2005; Levine, 2005; Murphy, 2006; Young, 2002). 

2. In administrator credential programs across the country, field experiences and practicum activities 
have become central components of the curriculum. However, their design, rigor, duration, and qual¬ 
ity vary widely. Critics note that in far too many credential programs such activities are episodic, 
decontextualized, not comprehensive, and poorly supervised (Hess & Kelly, 2005; Levine, 2007). As a 
result, candidates often receive fragmented and often inchoate perspectives about administrative roles, 
responsibilities, and problem solving scenarios. A deeper understanding of how job-related experiences 
shape and influence leadership expertise would be of great value to those who design and supervise 
field-based activities (Bottoms & O’Neill, 2001; Davis, Darling-Hammond, LaPointe, & Meyerson, 
2005; Hale & Moorman, 2003). 

3. Relatively little is known about how the types of professional experiences common to school leaders 
influence and/or stimulate transformative learning, cognitive skills, and the development of leadership 
skills. A deeper understanding of how workplace cognition influences leadership development is impor¬ 
tant for researchers and university faculty as well as district office administrators who are responsible 
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for the supervision, development, and evaluation of school principals (Borg, 2006; Hale & Moorman, 
2003; Mezirow, 1991; Usdan, McCloud, & Podmostko, 2000). 

4. A more fulsome understanding of how social context can influence leadership development would allow 
principal developers, mentors, and supervisors to more effectively align the characteristics of leadership 
candidates with the needs of particular schools, address the developmental learning needs of school 
leadership candidates, and enhance the pace of urban school reform efforts. 


5 METHODOLOGY 

5.1 Overview of the Study 

While we recognize that the majority of American school districts are located in various suburban or rural 
environments, the intense scrutiny and concern levied by policy makers and scholars compelled us to limit 
our sample to urban districts exclusively. 

To examine our three research questions, we developed an exploratory, ex post facto study that combined 
an online survey (see Appendix A) and follow-up telephone interviews with a randomly selected group of 
American urban school principals (Creswell, 2007). The survey portion of our study was completed in the 
autumn of 2008 and the follow-up interviews were completed in the spring of 2009. Survey items were derived 
from the 1996 ISLLC Standards, while interviews were conducted with a sub-sample of randomly selected 
survey participants using a standardized open-ended interview approach (Patton, 1990). 

5.2 Sample Selection: Procedures and Participants 

For the survey we randomly selected 200 principals from the largest K-12 school districts in 25 of America’s 
largest metropolitan areas as identified by the US Census Bureau County and City Data Book (2005). The 
calculation of sample size was based on an estimated response rate of 50%. The population proportion was 
set at a 95% confidence level and a 10%i confidence interval. By using the most conservative estimates for p 
and q (i.e., .50 and 1 - .50), the minimum sample size required was 96 (Hamilton, 1996). A minimum of 192 
surveys needed to be distributed to attain an estimated return rate of 50%. An additional eight surveys were 
distributed to provide a margin of safety should the return rate fail to reach 50% (surveys were distributed 
via an email link to Survey Monkey, an online survey service). Invitations to participate (or to decline to 
participate) in the study were sent via email to each principal in our sample of eligible schools. The survey 
design and distribution procedures were patterned after the Tailored Design Method by Dillman, Christian, 
and Smyth (2008). 

Using a stratified random sample selection process, eight schools from each large urban district were 
selected in groups proportionate to the number of elementary schools (grades K-6), middle schools (grades 
6-9), and high schools (grades 9-12) in that district. To provide a more parsimonious distinction between 
elementary schools, middle schools, and high schools we excluded all other grade level configurations from 
our population sample (e.g., K-8 or 6-12, etc.). To be eligible for participation in our study, each school 
needed to have a designated principal or a licensed administrator who served in an equivalent role. 
Response Rate 

A total of 104 (52%) surveys were returned. Three incomplete surveys were removed from the final 
sample. One hundred and one (101) completed surveys were used in our final analysis, which represented 
50.5% of the surveys distributed. Sixty-four percent of usable responses came from elementary principals, 
13% from middle school respondents, and 23% from high school respondents. Because our analysis did not 
account for non-response errors, particular subgroup survey responses may over or under-represented. 

5.3 Instrumentation 

Quantitative data were derived through the implementation and analysis of a two-part online survey (see 
Appendix A). In part one, respondents provided descriptive information about their gender, age, years of 
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leadership experience, graduate training, type of school/workplace, highest academic degree, and school 
demographic data. There were 12 items in part one. Part two consisted of 41-items constructed from the 
184 indicators of effective leadership practice contained in the six ISLLC standards (1996). The key purpose 
was to assess how and where principals acquired the leadership competencies contained in the standards. 
Because a survey with 184 items would be excessively lengthy, we organized the indicators around common 
themes and related concepts to produce a more manageable set of 41 items. 

By squeezing and combining the 184 ISLLC statements into a more manageable format, we ran the risk 
of altering or blurring the meaning of each statement and its key terms. We attempted to reduce such 
problems by, a) adhering closely to the language in the ISLLC standards, b) clustering concepts only within 
each standard (thereby maintaining the integrity of the standard), and c) pilot testing the survey with 
professors and practicing principals. 

Rather than posing items as questions, we wrote them as learning tasks. For example, the first item on 
the survey was written as Task 1: Learning how to develop and implement a long-term strategic plan for 
your school. This corresponded to language in the knowledge and skills subsections of ISLLC Standard 1 
(e.g., “.. .developing and implementing strategic plans” and “.. .an implementation plan is developed”). The 
concept of “long term” was added upon the advice of pilot-test subjects. Respondents were asked to rate the 
importance of their “administrative credential program in developing this skill” and the importance of their 
“on-the-job experiences in developing this skill.” Each of the 41 items was crafted using this method. 

Early drafts of the survey were piloted with 10 educational administration faculty members and 10 
practicing principals. Subsequent revisions and refinements were made based on their feedback. In cases 
where the ISLLC Standards have been used as the basis for administrator assessment instruments, the 
psychometric properties have been consistently strong. The recent development of the Vanderbilt Assessment 
of Leadership in Education (VAL-ED) is an example of this (Porter, Goldring, Murphy, Elliott, & Cravens, 
2006). Also, the widespread adoption of the ISLLC Standards by state credentialing and accreditation 
agencies underscores its prominence as a framework for administrator licensure in this country (Murphy, 
2003). As a result we did not conduct independent tests for validity or reliability. However, we recognize 
that the ability to generalize the findings of our study would be strengthened by such testing. 

For each of the 41 learning tasks, respondents were asked to choose from a five point Likert-type scale 
to rate two sub-items, a) the importance of their administrative credential program in the development of 
leadership expertise on that task, and b) the importance of on-the-job experiences in the development of 
leadership expertise on that task. The Likert scale choices included “unimportant,” “slightly important,” 
“moderately important,” “important,” and “very important.” To assess the impact of the ISLLC standards 
on the field, respondents were also asked to rate the overall importance of each learning task to effective 
school leadership on a scale of zero (0 = unimportant) to one (1 = very important). Choice options were 
provided in increments of .10. 

5.4 Survey Data Analysis 

Subgroups 

Statistical analyses were conducted for four subgroups identified from part I of our survey, 1) years of 
experience as a principal, 2) gender, and 3) school type. These three factors are especially prominent in the 
literature on leadership development (Paloniemi, 2006; Ruderman & Ohlott, 2002; Van Velsen & Guthrie, 
1998). Following is more descriptive rationale for focusing our analysis on these variables. 

1. Years of Experience. What principals learn on the job, and how, appears to be influenced by their 
level of experience. To assess this, we compared responses from principals who had one year of experience 
(as principals) with those who had ten years. As principals gain experience, they draw from an increasingly 
sophisticated reservoir of mental images, schemas, self-reflections, problem solutions, philosophical perspec¬ 
tives, and worldviews. In addition, we wanted to explore how experience influences principal perceptions of 
what they learned from their credential programs and on the job (Davis & Davis, 2003; Dreyfus & Dreyfus, 
1986). Finally, we assumed that a principal’s exposure to the 1996 ISLLC standards would be mitigated by 
his/her years of experience on the job (e.g., more experienced principals were less likely to have participated 
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in ISLLC based credential programs than less experienced principals). We acknowledge that there may be 
good arguments in support of other comparisons (e.g., 3 years vs. 5 or more years, or less than 5 years vs. 
6 or more years, etc.). However, we know of no empirical rationale for choosing one over the other. Our 
goal in comparing principals with one year and ten years of experience was to ensure sufficient variation 
in experience to tease out the potential influences of the ISLLC standards, proximity to graduate school 
experiences, and the job itself. 

2. Gender. As with experience, a strong empirical rationale exists for studying the role of gender 
in leadership development and learning among school principals. According to Owens (1995, p. 138), 
“.. .gender is the most influential factor in determining one’s view of the world and how one responds to what 
is perceived.” Ruderman & Ohlott (2002) maintain that in organizations men are exposed to more significant 
leadership opportunities than women. Although gender has little impact on leadership effectiveness, it may 
influence leadership style and behavior (Hopkins, O’Neil, Passarelli, & Bilimoria, 2008). 

3. School Type. What leaders do, how they learn, and what they learn can be influenced by different 
task environments (Daft, 2001). Differences in elementary and secondary school principal roles and responsi¬ 
bilities, behaviors, and leadership styles have been well documented (Davis, 1998). Our goal was to examine 
the influence of school level on leadership development. 

Statistical Tests 

A variety of statistical tests were used to analyze both whole-group and sub-group responses to survey 
items. First, demographic information was captured using totals, means, frequencies, percentages, and rank¬ 
ings. Second, to assess whole-group differences in responses to credential program and on-the-job response 
categories, we used a one-way analysis of variance and a principal component factor analysis was used to 
reduce and cluster the 41 learning tasks around key themes (factors). The results of the factor analysis were 
particularly useful in helping us to better understand patterns and interrelated features of the ISLLC tasks 
and their relative importance to principals. Third, learning task responses between sub-groups (e.g., expe¬ 
rience levels, gender, school types, credential program types) were compared using an independent samples 
t-test. For all tests, p values were set at p < .05. 

Respondents also rated the importance of each learning task to the field of educational administration 
using a fractional scale of 0 to 1.0 (e.g., 0 = unimportant, 1.0 = very important). First, we ranked the top 
and bottom learning task ratings for all respondents and each sub-group. Second, a one-way ANOVA was 
used to compare mean task ratings between subgroups (p < .05). For the sake of economy, we reported only 
the five highest and lowest rated tasks. 

5.5 Follow Up Interviews 

Follow up telephone interviews were held with 20 principals randomly selected from the 101 survey respon¬ 
dents. Interviewees consisted of 16 female and four male principals from 14 elementary schools and six 
secondary schools within the following metropolitan areas: Baltimore (2), Cincinnati (1), Cleveland (2), 
Dallas (2), Detroit (1), Los Angeles (2), Miami-Dade (1), Phoenix (1), Pittsburgh (4), Riverside (2), San 
Francisco (1), New York (1). 

Each phone interview took twenty to thirty minutes and included seven questions designed to prompt 
self-reflection, emotional content, and descriptive narratives. Using a semi-structured approach, the inter¬ 
viewer encouraged respondents to expand upon each question (e.g., allow it to trigger deep feelings, related 
experiences, and the construction of personal meaning). Although the interviewer asked the same questions 
of each subject and took detailed notes of the responses, the interviews were conducted in a conversational 
manner to provide flexibility for unstructured follow-up probes and inquiries. The seven questions included: 

1. What on-the-job experiences as an administrator have been most important in helping you develop 
your leadership skills? 

2. During your first years on the job as an administrator were there missing experiences that you wished 
you had? 

3. What kinds of on-the-job experiences do you believe are most important in developing a new principal’s 
leadership skills (e.g., considering a newly assigned, first time principal? 


http: //cnx.org/content/m45760/1.3/ 



Connexions module: m45760 


9 


4. What have you seen veteran principals do on the job to revitalize and reenergize themselves later in 
their careers? 

5. In what ways has your district enhanced/supported your leadership skills or facilitated on-the-job 
learning (coaching, professional development, conference attendance, principals’ meetings? 

6. As you look back, what learning activities in your administrative credential program were especially 
important in helping you to develop practical leadership skills? 

7. In what ways could your administrative credential program have better prepared you? 


note: These seven questions were field tested with several site principals and revised in response 
to their feedback and suggestions. 

Through inductive analysis common themes and concepts were identified, categorized, and analyzed (Patton, 
1990). Interview data were sorted according to Yukl, Gordon, and Taber’s (2002) Hierarchical Taxonomy of 
Leadership Behavior. Their framework, which is constructed upon a half-century of behavioral research on 
leadership, posits three meta-categories: relations, task, and change behaviors. Our purpose was to anchor 
and interpret interview responses around a well-established theoretical framework. 

Interview Data Analysis 

We began the analysis of interview data by searching for responses that contained “recurring regularities,” 
or common experiences and perceptions. These were identified and organized into one of the three meta¬ 
categories described by Yukl, et al. (2002). This process allowed us to interpret the interview data according 
to “sensitizing concepts” (Patton, 1990, p. 391). Within each meta-category, responses were further reduced 
through the identification of key phrases and key words. These were clustered by theme. Patton (1990, 
p.403) refers to this process as “convergence.” Within each theme a progressive synthesizing and narrowing 
process allowed us to construct holistic (and in some cases heuristic) descriptions by illuminating both the 
emotional and conceptual content of subjects’ responses. 

Finally, we examined responses that were unique to a particular subject and not part of the broader 
patterns found across interviews. The content of each response of this type was sorted by relevance. Content 
that was superfluous to the topic of leadership development was discarded. Content that was germane to the 
topic was retained. These responses often provided interesting perspectives about the leadership development 
experience. In some cases they raised important questions or issues for future research studies. 

6 RESULTS 

note: To assist the reader, we provide a list of the 41 learning tasks in Appendix A. 


6.1 Online Survey Findings 

We organized the reporting of survey results around respondent demographic data and focal research ques¬ 
tions number two (How important are on-the-job experiences and graduate credential programs in the de¬ 
velopment of leadership expertise among urban principals?) and number three (How important are the 1996 
ISLLC Standards for School Leaders in the development of leadership expertise among urban principals?) 
Research question one (How do urban principals develop leadership expertise?) is addressed more holistically 
in the discussion section of the manuscript. 

6.2 Respondent Demographic Data 

A slight majority (52.4%) of respondents had six or more years as a principal, while 47.6% had five or 
fewer years of experience. Seventy-six respondents (74.8%) were female and 25 (25.2%) were male. Thirteen 
(12.9%) respondents fell within the 25 to 45 year old age range, while 88 (87%) were over the age of 45. Sixty- 
nine (68.3%) respondents served as elementary (grades K-6) principals and 32 (31.7%) served as secondary 
principals (grades 6-12). 
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6.3 How important are on-the-job experiences and graduate credential programs in the devel¬ 
opment of leadership expertise among urban principals? 

Whole Group Results 

When all 101 responses were analyzed in the aggregate, on-the-job experience was rated as only slightly 
more important than college credential programs for each of the 41 learning tasks (4.38 versus 4.07 on a five 
point scale). 

A principal component factor analysis of responses to on-the-job experiences and credential programs 
reduced the 41 learning tasks to those that most strongly explained the phenomenon under analysis (e.g., 
what leaders learn and where they learn them). For on-the-job responses, fifteen learning tasks clustered 
around three factors defined as, 1) teaching and learning, 2) operations and resources, and 3) relations. We 
established a conservative factor-loading threshold of .500. Table 1 exhibits the three factors, their associated 
learning tasks, and factor loadings for responses to on-the-job experiences. 

Table 1 

Factor Analysis of 41 Learning Tasks: On-tlie-Job Experiences 

Task Number_Description_Factor loading_ 

Teaching and Learning 


15 

Coaching and supporting teachers 

.792 

34 

Promote teacher collaboration 

.760 

23 

Facilitate ongoing school improvement 

.722 

21 

Implement teacher professional growth 

.650 

17 

Use formative evaluation strategies 

.628 

39 

Promote principles of equity, diversity 
Operations and Resources 

.592 

27 

Manage legal requirements 

.734 

25 

Manage fiscal resources 

.715 

24 

Establish a safe learning environment 

.664 

28 

Use administrative technologies 

.644 

12 

Develop programs and procedures 
Relations 

.641 

3 

Practice self-reflection 

.716 

4 

Communicate with stakeholders 

.689 

32 

Work with community trends and issues 

.584 

20 

Positive community relationships 

.524 


For credential programs, fourteen learning tasks were clustered around the same three factors. Table 2 
contains the three factors, their associated learning tasks, and factor loadings. 

Table 2 

Factor Analysis of 41 Learning Tasks: Credential Programs 
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Task Number _ Description _ Factor loading 

Teaching and Learning 


17 

Use formative evaluation strategies 

.752 

13 

Apply adult motivational strategies 

.723 

34 

Promote teacher collaboration 

.701 

15 

Coach and support teachers 

.626 

21 

Implement teacher professional growth 
Operations and Resources 

.592 

25 

Manage fiscal resources 

.756 

29 

Work with bargaining agreements 

.730 

26 

Manage school facilities 

.670 

27 

Manage legal requirements 

.633 

33 

Build effective media relations 

.616 

31 

Access community resources 

Relations 

.615 

6 

Build group consensus 

.748 

5 

Dealing with competing interests 

.676 

11 

Participatory decision-making 

.560 


Subgroup Results 

As with the whole group, sub-group responses favored on-the-job experiences. However, there were subtle 
(but few significant) differences between sub-groups. For example, the importance of on-the-job experiences 
was slightly greater among more experienced principals and females. Interestingly, the importance of cre¬ 
dential programs was also slightly greater among more experienced and female principals, as well as for high 
school principals and principals who earned administrative credentials through private universities. Impor¬ 
tantly, the areas of difference between subgroups help to illuminate aspects of principal preparation and 
development that are not as well served by credential programs as they are by hands-on experiences in the 
workplace. 

On-the-Job vs. Credential Programs: Years of Experience 

Our first analysis examined the year in which respondents received their administrative credentials. Fifty 
(49.5%) respondents earned administrative credentials prior to the release of the ISLLC Standards in 1996. In 
contrast, 45 (44.6%) received credentials after 1996, and 15 (14.8%) received credentials after the adoption of 
No Child Left Behind in 2001. As a result, a comparatively modest number of respondents likely participated 
in ISLLC-based administrator credential programs. 

An independent samples t-test was used to compare responses to on-the-job experiences and credential 
programs among principals with one and ten years of experience. For on-the-job experiences, experienced 
principals were significantly more likely to give the highest ratings to two learning tasks (3 and 12). For 
credential programs, length of experience mattered with four tasks (5, 6, 11, and 39). Table 3 highlights the 
significant differences between principals with one and ten years of experience for both response categories. 

Table 3 

Independent Samples T-test: Less experienced versus More Experienced Principals 
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Task Number and Description 

Mean rating 

1 year exp. 

Mean rating 

10 years exp. 

Alpha 


On-the-Job Experiences 


3 

Practice self-reflection 

4.13 

4.70 

.037 

12 

Develop programs and procedures 

3.88 

4.48 

.029 


Credential Programs 


5 

Dealing with competing interests 

2.88 

4.13 

.034 

6 

Build group consensus 

3.63 

4.63 

.047 

11 

Participatory' decision-making 

3.13 

4.38 

.02 

39 

Promote equity and diversity 

3.38 

4.25 

.03 


On-the-Job vs. Credential Programs: Gender 

T-test results of the responses by male and female principals to the on-the-job experience category 
revealed four significant differences (tasks 10, 29, 30, and 34). In each case, females were significantly more 
likely to give higher ratings to job experiences. Similarly, for credential programs, females gave significantly 
higher ratings on three tasks (tasks 27, 34, and 35). Table 4 highlights the differences between male and 
female principals for both response categories. 

Table 4 

Independent Samples T-test: Male versus Female Principals 


Task Number and Description 

Mean rating 
Males 

Mean rating 
Females 

Alpha 


On-the-Job Experiences 


10 

Align programs with shared vision 

4.45 

4.76 

.029 

29 

Work with bargaining agreements 

3.20 

3.91 

.028 

30 

Promote culture of responsibility 

4.36 

4.74 

.012 

34 

Promote teacher collaboration 

4.27 

4.68 

.009 


Credential Programs 


27 

Manage legal requirements 

3.23 

4.03 

.005 

34 

Promote teacher collaboration 

3.45 

4.00 

.047 

35 

Develop code of ethics 

3.55 

4.24 

.003 


On-the-Joh vs. Credential Programs: School Type 

In general, elementary principals (K-6) found on-the-job experiences most useful, while secondary prin¬ 
cipals (6-12) more frequently favored credential programs. T-test results revealed only a few significant 
differences. Elementary principals were more likely to give high ratings to on-the-job experiences for tasks 
14 and 36 and to credential programs for tasks 13 and 34, while secondary principals gave higher ratings to 
task 5. Table 5 highlights the significant differences between elementary and secondary principals for both 
response categories. 

Table 5 

Independent Samples T-test: Elementary versus Secondary Principals 
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Mean rating Mean rating 

Task Number and Description _ Elementary Secondary _ Alpha 

On-the-Job Experiences 


14 

Implement standards-based 

3.34 

4.45 

.024 

36 

Work productively with people 3.61 

Credential Programs 

4.30 

.05 

5 

Dealing with competing interests 

3.35 

3.85 

.047 

13 

Apply adult motivational strategies 

4.36 

4.05 

.05 

34 

Promote teacher collaboration 

4.63 

4.12 

.05 


6.4 How important are the 1996 ISLLC Standards for School Leaders in the development of 
leadership expertise among urban principals? 

In terms of their importance to effective school leadership, the ISLLC-based learning tasks fared quite well. 
Thirty-two (78%) learning tasks were rated above the 80th percentile (on a scale of 0 to 1.0). By this 
measure, it appears that most learning tasks were important to the development of leadership expertise. In 
table 6 we list the top five and bottom five rated tasks and their mean values. 

Table 6 

The Five Most Important and Five Least Important ISLLC Learning Tasks 


Task number and description 


Mean 


Most Important 

2 Use data to improve teaching and learning .967 

10 Align programs with a shared vision .966 

15 Coaching and support teachers .963 

18 Sustain instruction-focused school culture .962 

9 Facilitate a shared vision for the school .943 

Least Important 

41 Identify broad political issues and influences .734 

33 Build/maintain effective media relationships .755 

29 Work effectively with bargaining agreements .759 

40 Advocate for schools in local/state/national arenas .762 

8 Learn to use symbolic leadership .820 


Importance of Learning Tasks: Subgroup Analysis 

Using a one-way analysis of variance test, a comparison of means by years of experience found that 
experienced principals were significantly more likely to give high ratings to task 29. For 16 (39%) of the 
41 learning tasks, females gave significantly higher ratings than males. Finally, analyses by school level and 
credential program type revealed no significant differences in response patterns. 

Table 7 highlights five of the most highly rated learning tasks by female respondents, the comparative 
mean scores between males and females, and their level of significance. 

Table 7 

Five Most Highly Rated Learning Tasks by Females (with Male Comparisons) 
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Task number and description 

Mean rating 
Female 

Mean rating 
Male 

Alpha 

25 

Manage fiscal resources 

.967 

.868 

.000 

10 

Align programs with shared vision 

.981 

.918 

.001 

18 

Sustain instruction-focused culture 

.978 

.909 

.001 

15 

Coach and support teachers 

.976 

.919 

.002 

5 

Dealing with competing interests 

.957 

.861 

.002 


Note: Males gave tasks 2, 15, 10, 18, and 21 their highest ratings (not significant) 


6.5 What matters most, on-the-job experiences or college credential programs, and with what 
skills? 

The survey data reveal that on-the-job experiences and college credential programs are both important 
sources of learning for school leaders. With an average rating of 4.38 on a five point Likert-type scale, it is 
clear that on-the-job learning experiences are highly regarded by school principals as sources of leadership 
skill development. However, an average rating score of 4.07 underscores a strong regard for the value of college 
credential programs as well. An important question then, is, with which learning tasks do job experiences 
and credential programs matter most? Conversely, which learning tasks are not learned as well on the job 
or in credential programs? It is to these questions that we now direct the analysis. 

As Table 8 illustrates, the job appears to be a better venue for learning how to sustain an instructionally- 
focused school culture, negotiate effectively with people who have competing interests, develop a strategic 
plan for the school, and use data to promote teaching and learning. Credential programs appear to be 
particularly useful as venues for learning how to develop a code of ethics, promote equity and diversity, coach 
and support teachers, and evaluate teachers. Finally, the job is less useful in teaching how to manage political 
tasks and influences, school advocacy strategies, and symbolic leadership, while credential programs are 
less useful in teaching media relations, collective bargaining, school facilities management, school advocacy 
strategies, and how to access community resources. 

Table 8 

Top 5 Ranked Learning Tasks: On-the-Job and College Credential Programs 


Mean 

On-the-Job Experiences 

Task number and description 

Mean 

Credential Programs 

Task number and description 

4.93 

18: Sustain instruction-focused culture 

4.18 

35: Develop code of ethics 

4.86 

5: Deal with competing interests 

4.13 

39: Promote equity and diversity 

4.85 

1: Develop long-term strategic plan 

4.12 

15: Coach and support teachers 

4.85 

2: Use data to promote teaching’learmng 

4.09 

16: Evaluate teacher performance 

4.75 

15: Coach and support teachers 

4.08 

18: Sustain instruction-focus culture 


Bottom 5 Ranked Learning Tasks 


On-the-Job Experiences 


Credential Programs 

3.58 

41: Political issues and influences 

3.04 

33: Build media relationships 

3.73 

40: Advocate for schools (political) 

3.09 

29: Work with bargaining agreement 

3.77 

33: Build media relationships 

3.20 

26: Manage school facilities 

3.81 

29: Work with bargaining agreement 

3.20 

40: Advocate for schools 

3.82 

8: Use symbolic leadership 

3.25 

31: Access community resources 
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6.6 Follow-up Telephone Interview Results 

Overview of Major Themes 

In the analysis of interview responses, our intent was to extend beyond survey responses and to identify 
common themes regarding perceptions about the development of leadership expertise. These themes are 
transcendent and apply to both college credential programs and on-the-job experiences. Five common 
themes emerged under Yukl’s, et al. (2002) “relations behavior” category. 

1. The importance of shadowing practicing administrators. 

2. Developing a professional network of principal colleagues for support and mentoring. 

3. Valuing the contributions of each member of the school’s certificated and classified staff. 

4. Understanding that the members of high functioning schools operate as teams. 

5. Developing a personal commitment to ongoing personal and professional growth, new learning, and 
goal directed behaviors. 

Four key themes emerged under the “task behaviors” category (Yukl, et. al, 2002). The first three pertain 
to both college credential programs and on-the-job experiences and their impact on the development of 
leadership expertise. 

1. More direct experiences with school budget preparation. 

2. Increased emphasis on the supervision and evaluation of teachers. 

3. More experiences using different venues, strategies, and styles of communication with school staff 
members and the community. 

4. College credential programs that place greater emphasis on the practical needs, issues, and tasks of 
school administration and less emphasis on theory. 

Under Yukl’s (2002) category of “change behavior” five common themes emerged. 

1. Executive coaching during one’s early years in administration to develop confidence in one’s professional 
knowledge and ability to promote change across the organization. 

2. An awareness of, and sensitivity to, the importance of school culture in the development and imple¬ 
mentation of successful organizational change efforts. 

3. District sponsored trainings and professional development activities. 

4. Having a district office supervisor who supported ongoing professional development. 

5. Working with a professional learning community of principals to promote ongoing growth and support 
and the exchange of useful ideas. 

As we coded and organized interview data, we often experienced difficulty in sorting data into the most 
appropriate category. For example, understanding that the members of high functioning schools operate as 
teams is essential to maintaining positive relationships, while it is also an essential component of change 
behavior, and a mechanism for structuring the accomplishment of workplace tasks. As a result, we placed 
themes in categories where the conceptual “fit” was strongest, but not necessarily exclusive. 

Interview Questions: Common Themes 

In a second level of analysis, we organized and synthesized responses according to their relationship to 
key interview questions. Again, we used the three meta-categories of leadership behaviors as a framework 
for organizing information and sharpening the analysis (Yukl, et ah, 2002). 

Important on-the-job experiences in the development of leadership expertise. Numerous 
respondents expressed the importance of the job in helping to build and maintain positive relationships. 
Some described this in very general terms, while others used specific phrases like “being available to staff and 
being visible with people,” “learning to stop and listen,” “listening to diverse needs,” treating people fairly,” 
and “learning not to be judgmental.” The job was also important in teaching principals to work productively 
with individuals or groups that provided important on-the-job experiences such as cluster meetings with 
other principals, mentoring by a veteran principal, or interactions with labor unions. 
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Several task experiences such as dealing with the complexities of a school budget and the supervision, 
evaluation and documentation of teachers were deemed particularly important for new principals. Other 
important task-related experiences included managing multiple priorities and tasks; opening and closing 
school; managing the district’s curriculum; learning how to assess teaching to improve student achievement; 
ensuring student safety, and a developing a systematic approach to promoting teaching and learning and 
managing school resources. 

Relating to change behaviors, respondents noted the importance of “learning the culture” and “thinking 
about the whole staff and unit” before setting a vision or assessing school needs. Several respondents also 
talked about strategies for dealing with difficult staff and “working with the naysayers who block the team.” 
Almost all respondents agreed that adopting an administrative disposition was an essential element of school 
change leadership. Finally, critical about the efficacy of college credential programs, some respondents stated 
that “there is nothing like sitting in the chair” and “almost everything I learned was on the job.” 

Missing on-the-job experiences during one’s first years. When asked about missing experiences 
during one’s first years, relationship issues continued to be most frequently cited by respondents. Some 
expressed regret at not “having a network of people you trust to call upon,” others noted that it took time to 
“forge deep and durable relationships, learn how to bring the community and parents together, implement 
shared decision making, and develop strong communication skills.” 

For task behavior one area was conspicuously problematic for new principals-inexperience in working 
with budgets. Other commonly cited missing experiences included, practice coaching teachers, analyzing 
instructional data, curriculum leadership, and training in public relations. Finally, learning how to manage 
change in a complex organization, developing “one’s own vision,” and how to “navigate the system” were 
acquired skills for most principals. As with many other aspects of leadership expertise, when it came to 
promoting school-wide change, new principals typically had very few experiences to draw upon. 

Important on-the-job experiences in developing a new principal’s leadership skills. Two 
relations based themes were strongly endorsed by several respondents, a) recognizing that everyone is im¬ 
portant to the successful accomplishment of the functions, mission, and goals of the school, and b) devel¬ 
oping a team culture. Building a supportive learning community among school stakeholders requires that 
new principals recognize the synergistic potential of the entire staff and promote collegiality, collaboration, 
communications, and mutual trust. Once again, the issue of budget management was prominent as was the 
supervision and evaluation of teachers. Other important task behaviors for new principals included contract 
management, student discipline, dealing with irate parents, and managing tough conversations over academic 
standards. 

Several respondents advised new principals to adopt thoughtful, yet cautionary, approaches to change 
management. Comments included, “look and listen,” “learn from strong teachers,” and “obtain knowledge of 
the climate and culture” before pursuing change initiatives. 

Respondents also provided a number of ancillary, yet constructive, suggestions for new principals. Among 
these were, “learn to trust and delegate,” “trust your instinct and judgments,” “follow-up with what you say,” 
“realize your style [and] self-reflect,” “not take things too personally,” and “reach out to the community.” 
Finally, having a mentor was widely recognized as an important experience for new principals. 

On-the-job learning: What veteran principals do to revitalize themselves. We asked veteran 
principals how they used on-the-job experiences to stimulate their own learning and professional development. 
Common relations-based themes included the importance of networking, sharing, and connecting with other 
principals, and adopting a service-oriented approach to the job. Almost every principal commented that 
being “really passionate, and in it for the kids,” and “staying involved with kids” helped to revitalize them. 

Task related strategies for professional revitalization included mentoring new principals, involvement in 
a professional association, taking advanced coursework, grant writing, district level committee participation, 
and other professional development training. 

In describing change behavior, Yukl’s (2002) leadership taxonomy identifies actions that encompass scan¬ 
ning the external environment, proposing new strategies, and/or encouraging innovative thinking. Seventeen 
of the 20 respondents endorsed one or more of these concepts. Respondents also noted the importance of, 
“setting higher goals for self and school,” “going for greater challenges,” and staying current with “your pro¬ 
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fessional development and professional readings.” Such comments underscore an achievement orientation 
present in many leaders and that is consistent with longstanding research on effective leadership (Yukl, 
1994). 

And finally, several respondents noted the importance of having a sense of humor, taking care of yourself 
outside of the job, and coming to grips with the fact that you can’t control the willful behaviors of others. 

How districts can support principal leadership through on-the-job learning. When asked to 
share specific actions or support systems present in their own districts that enhanced their leadership skills 
and learning, seven common responses surfaced. 

1. Thirteen respondents reported that their districts provided useful professional growth activities (e.g., 
“if the district doesn’t have it they find it,” and “they are great trainings”). 

2. Thirteen respondents cited their district’s support for professional conference attendance. 

3. Nine respondents mentioned specific activities that support their participation in professional learning 
communities with job-alike principals in their districts or across levels. Comments included, “principals 
meet as a learning community where there is lots of networking and sharing of strengths,” “we take 
learning walks...visit schools to learn from each other,” and “we have principal meetings by area with 
book studies and sharing of strategies and trends.” 

4. Eight respondents specifically mentioned the support they received from a coach or mentor, often when 
they first became a principal in the district. Sometimes the coach was a retired principal, in other 
cases it was the assistant superintendent or an outside executive coach. 

5. Seven respondents specifically mentioned principals’ meetings as an important source of support (e.g., 
“they are practical and focused on instruction,” “we have relevant activities and deal with program 
issues” and, “principal meetings are filled with discussion, challenges and sharing”). 

6. Four respondents commented about the support they received from their district offices. Comments 
included, “they are more friendly to us than ever before,” “they push us to the next level,” “they design 
what works for us (in reference to principal training and support)” and, “they support creativity and 
innovativeness.” 

7. Several interviewees stated that the district’s evaluation system, goal and standard setting processes 
and the development of professional growth plans enhanced their leadership skills. 

Helpful Leadership Development Activities in Administrative Credential Programs. Strong 
credential programs were particularly helpful in providing opportunities to observe and shadow practicing 
administrators who could connect learning with real world situations and environments. Learning activities 
that combined “research and reality” (e.g., job embedded projects and fieldwork) were also useful, as was 
learning in a cohort setting and working in teams. As one principal aptly stated, “it takes teamwork to 
make the dream work.” Other helpful program activities included participation in an assessment center, 
projects on cultural awareness, and learning activities on how to manage difficult students, the curriculum, 
budgeting, outcome-based instruction, and accountability requirements. 

Several respondents shared negative feelings about their credential programs (e.g., it “was just another 
hoop to jump through,” “absolutely nothing was helpful...it was not practical”). One principal cynically 
stated, “no, it’s all on the job, they can’t really prepare you.” 

Suggestions for how Credential Programs Can Better Prepare Principals. Particularly impor¬ 
tant suggestions for improving administrative credential programs included, a shadowing or field experience 
(e.g., “sitting in the principal’s chair for a week or two”); providing budget and facilities management learn¬ 
ing activities and experiences; better training in school law and special education; more practical and less 
theoretical coursework, and community engagement strategies. Finally, despite the many recommendations 
offered by respondents, virtually each agreed with the proposition that “no matter what you do in preparation 
you must hit the ground running and learn on the job.” 
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7 DISCUSSION 

“Each principal develops his/her own theory about the role of the principal and themselves as principals.” 
This quote from D. J. McGough (2002, p. 6), underscores the difficulty in trying to establish causal propo¬ 
sitions about the development of professional expertise. Who we are and what we have yet to become are 
the products of an enormously complex array of experiences, personality characteristics, motivations, aspi¬ 
rations, skills, social environments, and dispositions. We become good at what we do through distinctively 
experienced learning processes, environments, and life altering events (Nohria, Lawrence, & Nohria, 2002). 

In addressing the key questions that framed this study, the findings provide useful insights, but not ir¬ 
refutable conclusions. As is often the case when researching the complexities of human behavior, inevitable 
limitations arise regarding research design, methodologies used, and the theoretical foundations upon which 
the research rests. For example, it is important to consider that the principalship is not the only admin¬ 
istrative position through which one develops leadership expertise. Most principals begin their careers as 
assistant principals and often have years in which to develop and hone their leadership skills. Our research 
did not control for this, and as a result the findings cannot account for the mediating influence of the 
assistant principalship on the development of a principal’s expertise. It is also true that learning from a 
credential program and on the job are not dichotomous activities. What one learns on the job, and how 
well, is undoubtedly influenced by one’s credential program experience. Moreover, not all principal prepa¬ 
ration programs are equivalent in terms of their design and modes of delivery. Similarly, the qualities and 
characteristics of on-the-job experiences are inherently variable and individually unique. Our findings do not 
account for such differences, and, consequently, cannot be confidently generalized to all urban principals. 
Finally, the findings from our research are based entirely on the self-reported perceptions of urban principals. 
No other data were collected nor were direct observations made to corroborate these accounts. Therefore, it 
is with caution and prudence that we offer the following analyses and interpretations. 

7.1 How Do Urban Principals Develop Leadership Expertise? 

While on-the-job experiences were more important to principals than credential program experiences, we 
cannot dismiss the fairly high ratings given to credential programs by survey respondents. Importantly, the 
survey results do not warrant a carte blanche condemnation of credential programs, and certainly not to the 
extent expressed by their most vocal critics. 

The telephone interviews painted a more complex, and subtle picture. First, when supported by mean¬ 
ingful district office professional development activities, the job provided a form of leverage that enhanced 
workplace learning and deepened new knowledge structures. Second, credential programs provided an intel¬ 
lectual and cognitive framework for making sense of workplace experiences. Although for some respondents 
the credential program provided a weak foundation for leadership development, for others the opposite was 
true. 

How one judges the quality of learning via credential programs versus on-the-job experiences is important. 
Even if principals learn more on the job, the quality of learning may not be better than that experienced 
in credential programs. Stanford professor James March explained that the closer in temporal proximity 
managers are to meaningful events, the more importance they attribute to them (March, 2009). Following 
this logic, with each passing year the knowledge, skills, and dispositions acquired in a credential program lose 
ground to the influence and impact of real world experiences on how principals behave and think. Our point 
is that exact parallels are impossible to make when comparing the relative importance of college credential 
programs and on-the-job experiences. Following March’s logic, one could reasonably predict that credential 
programs would only rarely exceed the importance of on-the-job experience in the development of leadership 
expertise. 

Administrator credential programs were never meant to replicate the workplace. At best they are designed 
to simulate workplace environments, develop important management skills, and reorient the professional 
dispositions of candidates (Hallinger & Bridges, 2007). Nevertheless, a core problem with many programs 
is that they are overly generic and designed to accommodate the needs of the profession at large (Davis, et 
ah, 2006). As a result, graduates’ perceptions of program relevance will inevitably vary in proportion to the 
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degree in which the knowledge, skills, and dispositions gained through participation in them align with the 
needs, contexts, and conditions of their employing school districts. 

The notion that more rigorous and innovative school leadership preparation programs can magically 
transform classroom teachers into practice-ready school turnaround specialists defies the logic of leadership 
expertise and its development (Dreyfus & Dreyfus, 1986; Gladwell, 2008). This leads us to a disquieting 
dilemma in the comparative analysis of college credential programs and on-the-job experiences—how should 
we weigh the relative importance of each venue in teaching people to lead schools? 

Mezirow’s (1997) theory of transformative learning provides some clarity to this question. Professional 
development activities that are robust, hands-on, and that vigorously challenge a participant’s deeply held 
assumptions promote powerful new learning experiences. Such activities can occur in formally structured 
learning environments like credential programs or in more fluid and dynamic settings like the workplace. In 
either case, the transformative experience must be facilitated through a constructivist relationship between 
the learner and teacher (e.g., professor, site supervisor, or mentor). However, to optimize the potency of 
the workplace as a venue for professional development, learning activities must be thoughtfully structured 
upon personal reflections, feedback from supervisors, and dialogue with significant others (Evans, K., 2002; 
Fenwick, 2003). 

Our findings demonstrate that the 1996 ISLLC standards are widely perceived as valid and important 
benchmarks of administrative competence by urban principals. This conclusion is supported by the finding 
that 78% of the 41 ISLLC-based learning standards were considered to be important elements of adminis¬ 
trative practice by our respondents. Given the widespread adoption of these standards by administrative 
credential programs, the influence of these standards on professional learning has been positive and durable 
(Murphy, 2003). 

In the absence of a non-urban control group, our study was not designed to make sharp empirical distinc¬ 
tions between the perspectives of urban and non-urban principals. However, since each of our respondents 
was a principal in a large urban district, their perspectives and experiences are clearly pertinent to ur¬ 
ban school leadership. Nevertheless, additional research is needed to identify with precision the aspects of 
leadership development that are particularly germane to urban settings. 

7.1.1 The Relative Importance of Administrative Credential Programs and On-the-Job Expe¬ 
rience 

There are some important differences among principal subgroups in their perceptions about the comparative 
importance of credential programs and on-the-job experiences in the development of leadership expertise. 
In some cases we were able to propose empirically grounded explanations, in other cases we could only 
conjecture. 

Our first analysis examined the perceptions of inexperienced versus experienced principals. In keeping 
with the logic of March’s proposition about the salience of temporally proximal events, it was not surprising 
that experienced principals more frequently gave the highest ratings to on-the-job experiences than did 
less experienced principals. As noted above, these differences were statistically significant for only a few 
tasks. Interestingly, with more experienced principals, credential programs were a more important source 
of learning for tasks that fell within the “decision-making” category derived from our factor analysis (e.g., 
dealing with competing interests, building group consensus, facilitating participatory decision-making, and 
promoting equity and diversity). 

These differences speak to a generational gap between principals who were trained before and after the 
advent of No Child Left Behind. Prior to No Child Left Behind, the criticality of data-based decision making 
to school reform efforts was far less pronounced than it is today. In the current No Child Left Behind world, 
a much deeper emphasis has been placed on rational-logical educational structures, systems, processes, and 
decisions than ever before. Leadership skills requiring democratic forms of decision-making and relationship 
building may have lost ground to a more analytical orientation to school administration. 

Our second analysis examined the comparative ratings of credential programs and on-the-job experiences 
between genders. For female principals, on-the-job experience was a significantly more important source 
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for learning than for males in four task areas, a) how to align programs with a shared vision, b) how to 
work with collective bargaining agreements, c) promoting a culture of responsibility and accountability, and 
d) promoting teacher collaboration. Three of these tasks fall squarely within the category of developing 
relationships (these also align well with our factor analysis and with Yukl’s taxonomy). 

One possible explanation is that females may be more sensitive to social nuances and relationships than 
males. Since the essence of negotiation implies a degree of empathy and compromise, it is possible that 
females “read” their social environments better than men and are more effective at finding common ground 
among those with diverse backgrounds and/or competing interests (Gerson & Peiss, 1985). Of course, 
another implication is that males are predisposed to be comparatively weak in terms of conducting social 
relationships. Although we are not prepared to entertain such a conclusion, we suggest that credential 
programs could (and should) pay particular attention to learning activities that expose males to diverse 
social environments and that enhance their sensitivity to differences in perspectives among people with 
diverse backgrounds. 

As a group, female principals were more consistent than males in their regard for the importance of 
on-the-job experiences. On 37 of the 41 learning tasks, females gave the highest ratings to on-the-job 
experiences, whereas males gave the highest ratings to only 30 tasks. Ruderman and Ohlott (2002) offer 
a plausible explanation for this discrepancy. In many complex organizations, males are exposed to more 
significant learning opportunities than women. This phenomenon holds true at all levels of an organization’s 
employment hierarchy. If Ruderman and Ohlott’s proposition is correct, men who become school principals 
are more likely than women to have had a history of job-related exposures to administrative environments 
and activities as teachers and assistant principals. By the time many males become principals they have 
accumulated a larger repertoire of important administrative experiences than females. In contrast, females 
who become principals may be more likely to find greater novelty in the routine roles, and tasks of the job 
than do males. Once again, if March’s proposition about the salience of temporally proximal events is true, 
then the results of our gender analysis make more sense. 

Comparisons between elementary and secondary principals yielded two particularly noteworthy differ¬ 
ences. For secondary principals the job was a more important source of learning about how to implement 
a standards-based curriculum and working productively with people who have different values and beliefs. 
The political complexities of the urban American secondary school are legendary. Thus, the latter finding is 
not surprising. However, that the job is a particularly important source of learning for secondary principals 
on how to implement a standards-based curriculum may suggest that administrative credential programs are 
more effectively organized and structured around the elementary curricula. 

7.1.2 How Do National Standards for School Leadership Influence the Development of Lead¬ 
ership Expertise? 

A review of the five most highly ranked learning tasks illuminates what contemporary urban principals 
believe are the most relevant components of effective school leadership. All five fall within the category 
of “teaching and learning” and figure prominently in the literature on effective educational leadership and 
policy making. Moreover, as a body, these tasks encompass many core concepts commonly associated with 
instructional leadership: using data to improve teaching and learning, aligning programs and professional de¬ 
velopment with a clear vision, creating an instructionally-focused culture, coaching and supporting teachers, 
and establishing a shared vision of learning for the school (Murphy, 2002; Woolford & Hoy, 2008). 

Interestingly, four of the five lowest ranked learning tasks contained political aspects of the principalship 
that have long been associated with urban education (e.g., staying abreast of local, state, and national 
trends and issues; building positive media relations; working with collective bargaining agreements; political 
advocacy for schools). Similarly low rankings were found for learning tasks relating to managing facilities 
and grounds, working with communities, and being aware of useful community resources. 

One explanation for these low rankings is the intense focus today on instructional leadership generated 
by No Child Left Behind and on the growing body of research relating to leadership and student achievement 
(Darling-Hammond, et al. 2007). The mandates and sanctions contained within No Child Left Behind—as 
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well as numerous state reform policies—have virtually assured the narrowing of principals’ attentions. We are 
not prepared to comment on whether the refocused roles of school leaders have resulted in better teaching 
and learning. However, eight years after the implementation of No Child Left Behind, serious concerns 
remain about persistent achievement gaps, the persistent underachievement of minority students, and the 
capacity of public schools to engage in transformative reform (Hamilton, Steelier, & Yuan, 2009; Vanneman, 
Hamilton, Baldwin Anderson, & Rahman, 2009). 

Clearly, urban principals today are highly focused on roles and tasks related to the management of 
instruction, comparatively less focused on the traditional management components of the job, and held ac¬ 
countable for the academic performance of students and the professional practices of teachers. Moreover, 
they must ensure that the functions, goals, and performance outcomes of their schools are highly transparent 
and accessible to parents and community members. Our findings support current literature and policy imper¬ 
atives that portray the primary role of the principal as an instructional leader (Seashore Louis, Leithwood, 
Wahlstrom, & Anderson, 2010). 

Our findings also point to the unevenness of the 1996 ISLLC standards. Not all standards are considered 
equally important to urban principals, although ISLLC crafted them without preferential weightings. In 
addition, the recently revised ISLLC standards (2008) feature several criteria deemed relatively unimportant 
by contemporary urban principals. The new standards highlight the importance of effective management 
of school operations; collaborating with community members; working effectively with diverse communities; 
the ability to mobilize community resources, and keeping abreast of political, social, economic, legal, and 
cultural trends and contexts. However, the principals in our study found these to be less important than 
the criteria pertaining to instructional leadership. Granted, implicit (and in some cases explicit) within the 
revised ISLLC standards is the primary goal of promoting powerful teaching and learning. Nevertheless, our 
findings suggest a gap between the revised ISLLC standards and current practices and perspectives of urban 
principals. 

7.1.3 Lessons Learned From the Follow up Interviews 

The results of the interviews provided depth and context regarding the factors that influence leadership 
development and practice. They also illuminated aspects of school leadership that are especially important 
to principals, and conversely, aspects that are comparatively unimportant. Finally, they provided insights 
that could be useful in the development and operation of administrator credential programs. 

Nevertheless, we found few discernable patterns among interview respondents regarding the factors 
deemed most important to leadership and/or leadership development. Responses covered several criteria 
contained in the 1996 and 2008 ISLLC standards (e.g., instructional leadership, collaborations, shared vi¬ 
sion, school culture, operations management, teacher evaluation, communications, ethical leadership, and 
leadership style). A common topic of concern was the failure of credential programs to provide sufficient 
training in budget management. As a consequence, budget management was learned on the job (essentially 
“trial by fire”). Likewise, for most respondents credential programs failed to provide a robust and useful 
practicum (e.g., fieldwork activities were not comprehensive, few opportunities were provided to shadow 
school administrators, mentoring relationships were weak-if they existed at all, and performance feedback 
was inconsistent and shallow). Also prominently missing (or only occasionally cited) from the interview 
responses were topics relating to student achievement data analysis, use of technology, student discipline, 
and self-reflection. 

However, the interview data pointed to a number of factors that were particularly helpful in the devel¬ 
opment of leadership expertise. We were impressed by the large number of principals who reported that 
district sponsored professional development activities have become increasingly important resources for their 
own professional growth and learning. They also described an increase in district support for professional 
learning communities that enabled new learning, problem solving, and the sharing of challenges and success¬ 
ful practices among principals. Throughout the interviews the commitment for (and pursuit of) professional 
growth and self-improvement was a consistent theme. Respondents were passionate about their jobs and 
deeply committed to making their schools better places for all children to learn. 
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Several respondents also spoke to the importance of their school district principal evaluation system in 
providing focus and direction for their own professional growth. Not surprising (at least to those of us who 
have been school principals), was the importance of having access to a network of colleagues who can provide 
professional and emotional support. 

An important objective of our study was to use interview responses to extend beyond the survey responses. 
Most notable was our finding that the development of leadership expertise was stimulated by college credential 
programs and honed through the unique intersection of professional experiences and personal characteristics. 
Together these elements provided both the grist and the impetus for the development of leadership expertise. 

7.1.4 The Larger Lessons From Our Study 

Theories of adult learning, experiential learning, transformative learning, and situated cognition have been 
part of the education research landscape for decades, and yet many of the basic concepts and principles 
regarding how adults best learn have been conspicuously missing from the administrative credential experi¬ 
ences of many principals. Both credential programs and on-the-job learning experiences have the potential to 
provide relevant and robust learning experiences that can advance professional leadership expertise. In the 
ideal, the two venues, when structured properly, can provide progressive and developmentally appropriate 
professional learning for school administrators. To do this well requires that both venues employ learning 
activities that are designed around seven key principles of adult learning (Knowles, Holton, & Swanson, 
2005; Mezirow, 1991). 

1. Learning activities are self-directed. 

2. Past experiences are used to understand and interpret new information. 

3. New information is important and relevant to them personally. 

4. They are provided with problem-based learning activities that require the integration of new knowledge, 
past experience, and problem-solving activities. 

5. They confront disorienting dilemmas that challenge deeply held perspectives, assumptions, beliefs, and 
world-views. 

6. Opportunities are provided to engage in professional discourse with peers around topics relevant to 
their jobs. 

7. They become progressively less dependent upon their teachers and more dependent upon their own 
powers of inquiry. 

Descriptions of credential programs by many respondents in our study do not closely match these principles 
of adult learning. In addition, the credential programs described by respondents were typically generic 
in structure (e.g., one-size-fits-all) and often perceived as irrelevant to the unique workplace demands of 
principals. 

Our findings also point to several suggestions for how credential programs and school districts can enhance 
the leadership skills of principals. When we compared the five most important learning tasks to the profession 
at-large with the five highest rated tasks under the categories of on-the-job job experiences and credential 
programs, we found several gaps between what tasks were learned, and where, and their overall importance 
to the profession. For example, the five learning tasks most important to urban principals were, a) using 
data to improve teaching and learning, b) aligning programs with a clear vision for the school, c) coaching 
and supporting teachers, d) establishing an instructionally focused school culture, and e) developing a shared 
vision of learning. However, the five tasks most likely to be learned in credential programs overlapped in 
only two cases coaching and supporting teachers, and establishing an instructionally focused school culture. 
Credential programs were not judged as being particularly good venues for learning how to use data to 
improve teaching and learning or aligning educational programs with, and developing, a shared vision for 
the school. Interestingly, whereas credential programs were deemed to be good places to learn how to 
promote the principles of equity and diversity, this task was not rated highly in terms of its importance to 
the profession. 
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In contrast, the five highest rated tasks under on-the-job experiences overlapped with three of the five 
most important tasks to the profession at large, a) using data to improve teaching and learning, b) coaching 
and supporting teachers, and c) promoting an instructionally focused culture. Job experiences fell short 
in only two areas deemed most important to the profession, a) aligning programs with a clear vision for 
the school, and b) developing a shared vision of learning. It is important to note that neither credential 
programs nor the workplace were highly rated as venues for learning how to manage budgets. These findings 
provide both credential faculty and superintendents with clear foci for developing more relevant training and 
professional development activities for urban principals. 

Credential programs must work harder (and smarter) to provide learning activities that are more sharply 
developed around contextually relevant workplace situations and environments. Although credential pro¬ 
grams will never supplant the importance of on-the-job experience in the development of leadership expertise, 
they can (and should) prime candidates for the complexities of professional practice by designing problem- 
based learning activities that treat the tasks and roles of school administration as highly relational and 
integrated. This will require a fundamental paradigm shift for many professors of educational administra¬ 
tion (e.g., away from being a “sage on the stage” and toward becoming a “guide on the side,” and away from 
treating the highly relational dimensions of administrative work as independent domains of knowledge). 

The potential to use on-the-job experiences as vehicles for professional growth and development has 
not been fully understood, appreciated, or utilized by school district administrators who supervise site 
principals. However, the interview results suggest that school districts can more effectively leverage on-the- 
job experiences to enhance principal expertise through the mechanisms of principal evaluation processes, 
the alignment of professional development activities with evidentiary data relating to school performance 
outcomes, by formalizing and supporting an ongoing professional network of principals, and by establishing 
and supporting professional learning communities (Darling Hammond, et. al, 2007). 

To conclude, it is unfortunately true that the closer one looks in the mirror, the more evident one’s 
blemishes become. This has certainly been the case with administrative credential programs in recent years. 
But blemishes notwithstanding, the increased scrutiny by researchers, policy makers, and practitioners of 
the structures, processes, and outcomes of administrator preparation programs in America has engendered 
a number of fresh ideas and practices in the field. We refer to the intriguing examples of innovation pro¬ 
vided by non-profit principal preparation programs like New Leaders for New Schools, school districts like 
New York City, and university programs like Delta State University. Programs like these have been in¬ 
formed (if not stimulated) by the experiential learning theories of researchers like David Kolb (1983), the 
transformative learning theories of Jack Mezirow (1997), the problem-based learning approaches pioneered 
by Philip Hallinger and Edwin Bridges (2007), and the school leadership development research by Joseph 
Murphy (2002), and others. The point is, that administrator preparation in America is in transition: away 
from the sclerotic, generic, and decontextualized academic practices of the past 50 years and toward a more 
organic, adaptive, and contextually focused endeavor steeped in the day-to-day realities and challenges of 
the workplace. 

Nevertheless, it is probably true that the relevance and impact of administrator credential programs will 
always be subordinate to on-the-job experiences. As a result, we gently remind our colleagues in the field (as 
well as policy makers) that even the most progressively designed and effectively executed credential program 
cannot be expected to produce “practice-ready” school principals equipped with a full repertoire of skills 
needed to transform public schools. The development of expertise requires time and practice (Dreyfus & 
Dreyfus, 1986; Gladwell, 2008). Granted, college programs can do (and in many cases are doing) a better 
job of constructing a solid conceptual scaffold for the development of effective school leaders, but at the end 
of the day, there is no substitute for experience. 

7.2 REFERENCES 

CLICK HERE to view REFERENCE SECTION 2 

2 See the file at <http://cnx.org/content/m45760/latest/Davis References.pdf> 
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7.3 APPENDIX A 

CLICK HERE to view APPENDIX A 3 


3 See the file at < http://cnx.org/content/m45760/latest/Davis Appendix A.pdf> 
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Appendix A 

Online Survey Questions 
Part I: Descriptive/Demographic Information 

1. How many years have you worked as a school principal? 

2. In what types of schools have your worked as a principal? For how many years? 

3. In what type of institution/program did you earn your administrative credential/license? 

4. Please indicate the name of the institution/program where you received your initial 

administrative credential/license. 

5. When was your initial administrative credential/license issued? 

6. What is your highest degree? 

7. What type of district do you currently work in? 

8. How many students are currently enrolled in your school? 

9. What is the approximate percentage of students who qualify for the Free and Reduced Lunch 

Program in your school? 

10. What percent of students in your school fall within the following ethnic profiles? (African 

American, Asian, Hispanic, Middle Eastern, Native American, Pacific Islander, White, 
other) 

11. Please indicate your gender. 

12. Please indicate your age group. 

Part II: ISLLC-Based Learning Tasks 

Task 1: Learning how to develop and implement a long-tenn strategic plan for your school. 

Task 2: Learning how to identify, collect, prioritize, and analyze data to promote powerful 
teaching and learning and to advance student achievement. 

Task 3: Learning how to practice deep and forthright self-reflection about your job perfonnance, 
behaviors, decisions, actions, values, and beliefs. 

Task 4: Learning how to communicate important infonnation about the school, its programs, 
processes, and accomplishments to multiple stakeholders. 

Task 5: Learning how to negotiate effectively with people who have diverse or competing 
interests. 

Task 6: Learning how to bring a group to consensus around a challenging decision. 

Task 7: Learning how to apply systems thinking in the management of school programs, change 
efforts, and operations. 

Task 8: Learning how to use symbolic leadership (e.g, rituals, ceremonies, specific behaviors) in 
the pursuit of important school goals. 

Task 9: Learning how to facilitate the development, articulation, implementation, and 

stewardship of a vision of learning that is shared and supported by the school community. 
Task 10: Learning how to align program development, planning and professional activities with 
a clearly articulated and shared vision for the school. 

Task 11: Learning how to involve multiple school stakeholders in important decisions, programs, 
and activities that matter for student learning. 
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Task 12: Learning how to apply important theories and concepts about learning in the planning 
and development of educational programs, teacher evaluation procedures, and staff 
development activities. 

Task 13: Learning how adult motivational strategies can be applied in the development of 
professional growth activities for teachers and staff and continuous school-wide 
improvement efforts. 

Task 14: Learning how to design, implement, evaluate, and refine a standards-based curriculum. 

Task 15: Learning how to apply the principles of effective instruction to coach and support 
teachers and to promote powerful teaching and learning. 

Task 16: Learning how to measure and evaluate teacher perfonnance according to empirically 
supported standards of effective teaching. 

Task 17: Learning how to use key principles of adult learning to inform and improve formative 
evaluation strategies and procedures. 

Task 18: Learning how to assess, nurture, and sustain a school culture and instructional program 
that is conducive to student learning and staff professional growth. 

Task 19: Learning how to use technology to improve teaching and learning. 

Task 20: Learning how to cultivate and maintain positive relationships with community leaders 
and groups through effective communications and active involvement. 

Task 21: Learning how to plan and implement professional growth activities for teachers and 

staff that are aligned with the teaching and learning goals of the school and student needs. 

Task 22: Learning how to use different decision-making strategies to address different situational 
demands. 

Task 23: Learning how to draw upon the principles of organizational development and behavior 
to plan and facilitate ongoing school-wide improvement and change processes. 

Task 24: Learning how to establish systems and procedures to ensure a safe and productive 
learning environment for students, teachers, and staff. 

Task 25: Learning how to manage fiscal resources accurately, legally, and effectively in ways 
that promote the educational goals of the school. 

Task 26: Learning how to manage school facilities and grounds in ways that promote the 
educational goals of the school and that are fiscally sound. 

Task 27: Learning how to manage the legal requirements of school operations and programs, 
personnel management, and student affairs. 

Task 28: Learning how to develop and manage administrative technologies to improve the 
management of student data and organizational systems. 

Task 29: Learning how to work effectively with collective bargaining agreements. 

Task 30: Learning how to promote a culture of shared responsibility, ownership, and personal 
accountability among all school stakeholders. 

Task 31: Learning how to access community resources that can improve teaching and learning. 

Task 32: Learning how to recognize and work successfully with shifting community trends, 
issues, and needs. 

Task 33: Learning how to build and maintain effective relationships with the media. 

Task 34: Learning how to promote professional collaborations among teachers and staff. 

Task 35: Learning how to develop and use a professional code of ethics and a philosophy of 
education to guide leadership behavior and decision-making. 

Task 36: Learning how to work productively with people who have values, beliefs, or ethical 
standards different from your own. 
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Task 37: Learning how to adopt the personal and professional orientations and dispositions 
necessary to lead effectively. 

Task 38: Learning how different models of school governance can work to advance the school’s 
vision, mission, goals, and powerful teaching and learning for all students. 

Task 39: Learning how to promote the principles of equity, respect for diversity, and the role of 
schooling in a democratic society among all school stakeholders. 

Task 40: Learning how to advocate for schools through local, state, and national political arenas. 

Task 41: Learning how to identify important national and global political, economic, and social 
trends and issues and their potential influence on public schools. 



